Coefficient of determination in nonlinear signal processing

For statistical design of an optimal "lter, it is probabilistically advantageous to employ a large number of observation random variables; however, estimation error increases with the number of variables, so that variables not contributing to the determination of the target variable can have a detrimental e!ect. In linear "ltering, determination involves the correlation coe$cients among the input and target variables. This paper discusses use of the more general coe$cient of determination in nonlinear "ltering. The determination coe$cient is de"ned in accordance with the degree to which a "lter estimates a target variable beyond the degree to which the target variable is estimated by its mean. Filter constraint decreases the coe$cient, but it also decreases estimation error in "lter design. Because situations in which the sample is relatively small in comparison with the number of observation variables are of salient interest, estimation of the determination coe$cient is considered in detail. One may be unable to obtain a good estimate of an optimal "lter, but can nonetheless use rough estimates of the coe$cient to "nd useful sets of observation variables. Since minimal-error estimation underlies determination, this material is at the interface of signal processing, computational learning, and pattern recognition. Several signal-processing factors impact application: the signal model, morphological operator representation, and desirable operator properties. In particular, the paper addresses the VC dimension of increasing operators in terms of their morphological kernel/basis representations. Two applications are considered: window size for restoring degraded binary images; "nding sets of genes that have signi"cant predictive capability relative to target genes in genomic regulation. ( 2000 Elsevier Science B.V. All rights reserved.